Apparatus, system, method and storage medium

ABSTRACT

A device is provided, comprising: an image storage unit which is configured to store image data obtained by a monitoring camera; an analyzing unit which is configured to perform image analysis on the image data and, in response to a detection of a target object to be detected among objects which are possibly included in the image data, generate feature data indicating a feature of the target object; and a transmitting unit which is configured to transmit metadata including the feature data and at least one of a shooting time of the image data in which the feature data is generated or an address of the image data within the image storage unit to a file server which is accessible to another apparatus different from the device.

The contents of the following Japanese patent application(s) areincorporated herein by reference:

2019-231460 filed in JP on Dec. 23, 2019

BACKGROUND 1. Technical Field

The present invention relates to a device, a system, a method, and astorage medium.

2. Related Art

For a conventional system which uses a monitoring camera to performmonitoring, various techniques which provide the access to image datashot by the monitoring camera are proposed (for example, see PatentLiterature 1).

Patent Literature 1: Japanese Unexamined Patent Application PublicationNo. 2015-162232

However, if the shot image data is simply stored, image analysis must beperformed on each of the image data each time in order to access theimage data which includes a desired object.

SUMMARY

To solve the above-described problem, a first aspect of the presentinvention provides a device. The device may include an image storageunit which stores image data obtained by a monitoring camera. The devicemay include an analyzing unit which performs image analysis on the imagedata and, in response to a detection of a target object to be detectedamong objects which are possibly included in the image data, generatesfeature data which indicates a feature of the target object. The devicemay include a transmitting unit which transmits the metadata includingthe feature data and at least one of the shooting time of the image datain which the feature data is generated or the address of the image datawithin the image storage unit to a file server accessible to anotherapparatus different from the device.

A second aspect of the present invention provides a system. The systemmay include a device in the first aspect. The system may include a fileserver which stores the metadata transmitted from the device. The systemmay include a monitoring terminal connected to the device and the fileserver. The monitoring terminal may have a selecting unit which selectsany one of the metadata stored in the file server in response to theoperation by the operator. The monitoring terminal may have a readingunit which reads from the device the image data corresponding to theselected metadata. The monitoring terminal may have a display controlunit which displays the image data read by the reading unit.

A third aspect of the present invention provides a method performed inthe device. The method may include an image storing step to store in thestorage device the image data obtained by the monitoring camera. Themethod may include an analyzing step to perform an image analysis on theimage data and, in response to a detection of the target object to bedetected among objects which are possibly included in the image data,generate the feature data indicating the feature of the target object.The method may include a transmitting step to transmit the metadataincluding the feature data and at least one of the shooting time of theimage data in which the feature data is generated or the address of theimage data within the storage device to a file server accessible toanother apparatus different from the device.

A fourth aspect of the present invention provides a storage medium whichrecords a program. The program may cause a computer of the device toserve as an image storage unit which stores the image data obtained bythe monitoring camera. The program may cause the computer of the deviceto serve as the analyzing unit which performs the image analysis on theimage data and, in response to a detection of a target object to bedetected among objects which are possibly included in the image data,generates the feature data which indicates the feature of the targetobject. The program may cause the computer of the device to serve as thetransmitting unit which transmits the metadata including the featuredata and at least one of the shooting time of the image data in whichthe feature data is generated or the address of the image data withinthe image storage unit to the file server accessible to anotherapparatus different from the device.

The summary clause does not necessarily describe all necessary featuresof the embodiments of the present invention. The present invention mayalso be a sub-combination of the features described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a system 1 according to an embodiment.

FIG. 2 shows a stored content of a shooting condition storage unit 62.

FIG. 3 shows a stored content of a detection target storage unit 63.

FIG. 4 shows an operation related to the delivery of the image analysisengine.

FIG. 5 shows an operation related to monitoring.

FIG. 6 shows an example of a computer 2200 in which a plurality ofaspects of the present invention may be entirely or partially embodied.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the present invention will be described through theembodiments of the invention. However, the embodiments described beloware not to limit the claimed invention. In addition, all of thecombinations of the features described in the embodiments are notnecessarily required in the solution of the invention.

[1. System 1]

FIG. 1 shows the system 1 according to the present embodiment.

The system 1 includes one or more monitoring cameras 3, one or moredevices 4, a file server 5, a delivery server 6, a monitoring terminal7, and one or more mobile terminals 8, and performs the remotemonitoring of the monitored region.

The monitored region may be a region monitored by the monitoringterminal 7 and may include the region shot by the monitoring camera 3(also referred to as a shot region). In the present embodiment, themonitored region is the premises of a facility as one example, but itmay also be a road, a walkway, or the like. The facility may be a plant,or may be a school, a house, a station, a museum, or the like. Examplesof the plant include an industrial plant such as a chemical plant or abiological plant, as well as a plant for managing and controlling awellhead or its surrounding area of a gas field, an oil field or thelike, a plant for managing and controlling power generation such ashydraulic power generation, thermal power generation and nuclear powergeneration, a plant for managing and controlling energy harvesting suchas solar photovoltaic generation, wind power generation or the like, anda plant for managing and controlling water and sewerage, a dam or thelike. The facility may be provided with variety of security sensors (notshown) besides the monitoring camera 3.

Herein, the device 4, the file server 5, the delivery server 6, themonitoring terminal 7, and the mobile terminal 8, among each componentof the system 1, may be connected with each other via a communicationnetwork 10. In the figure, the device 4, the file server 5, the deliveryserver 6, the monitoring terminal 7, and the mobile terminal 8 areconnected via the single communication network 10, but they may beconnected via separate communication networks.

The communication network 10 may be composed by including variousnetworks such as the Internet, Wide Area Network (WAN), and Local AreaNetwork, or the combination thereof. The communication network 10 mayinclude a connection point which uses at least one of wiredcommunication or wireless communication. The communication network 10may be achieved by a dedicated line separated from a public line such asthe Internet.

[1. 1 Monitoring Camera 3]

Each of the monitoring cameras 3 performs a shooting of the shot region.Each of the monitoring cameras 3 may supply to the device 4 the imagedata obtained through the shooting. The monitoring camera 3 may be acamera for a visible light or may be a camera for an infrared light oran ultraviolet light (as one example, X-ray).

[1. 2. Device 4]

Each of the devices 4 is connected to at least one of the monitoringcameras 3. Each of the devices 4 has an image storage unit 40, ananalyzing unit 41, and a communication unit 45.

[1. 2. 1. Image Storage Unit 40]

The image storage unit 40 stores the image data obtained by themonitoring camera 3. The image storage unit 40 may store each image dataobtained by the plurality of monitoring cameras 3.

The image storage unit 40 may store the image data shot by each of themonitoring cameras 3 in association with the date and time of theshooting and the identification information (also referred to as cameraID) of the monitoring camera 3. The image storage unit 40 may save theimage data for at least a reference period (as one example, five days)and may successively overwrite with the new image data. The capacity ofthe image storage unit 40 may be 1 TB as one example.

[1. 2. 2. Analyzing Unit 41]

The analyzing unit 41 performs an image analysis on the image data and,in response to a detection of the target object to be detected among theobjects which are possibly included in the image data, generates thefeature data indicating the feature of the target object. The analyzingunit 41 may perform the image analysis on the image data stored in theimage storage unit 40 and may supply the feature data to thecommunication unit 45. The analyzing unit 41 may perform the imageanalysis for each of the monitoring cameras 3 and generate the featuredata.

Note that the target object may be an object to be noticed for asecurity reason among the objects which can be detected through theimage analysis. For example, the target object may be at least one of ahuman face, a whole body of a human (as one example, an adult or achild), clothes, a vehicle (a vehicle in motion, a vehicle which isstopped), a car model, a car color, a license plate, a particular object(a bag, a weapon), or the like. In addition, the target object may be aface of a particular person or a particular vehicle. The target objectmay also be a person or a vehicle which moves across images (acrossframes) or may be an action performed across a plurality of images.

If the target object is a human face, the feature data may indicate thefeature of the face within the image. The feature of the face within theimage may be the positional relationship of the feature points of theface, may be the color of the pupil, the skin, or the hair, or may bethe color, shape, or the like of an accessory (as one example, glasses,earrings, or the like). Furthermore, if the target object is a vehicle,the feature data may indicate the car model of the vehicle within theimage. Furthermore, if the target object is the license plate, thefeature data may indicate the number of the vehicle within the image.

Note that the target object may be set for the device 4 by an operatorin advance or may be optionally changed. The number of types of thetarget objects may be one or may be multiple. If there is a plurality oftarget objects, the analyzing unit 41 may generate the feature data ofat least one of the target objects. The feature data may be used as anidentity to extract particular image data from the plurality of imagedata.

The analyzing unit 41 may perform the image analysis on the image datathrough one or more image analysis engines (not shown). As one example,in the present embodiment, the analyzing unit 41 may perform the imageanalysis by operating the plurality of image analysis engines inparallel or at different timings.

Operating the plurality of image analysis engines at different timingsmay mean, for example, operating one image analysis engine in a periodwhen the image analysis is performed on the image data shot during thedaytime and operating another image analysis engine in a period when theimage analysis is performed on the image data shot in the nighttime.Operating the plurality of image analysis engines at different timingsmay also mean operating one image analysis engine in a period when theimage analysis is performed on the image data shot with a follow lightand operating another image analysis engine in a period when the imageanalysis is performed on the image data shot with a back light.Operating the plurality of image analysis engines at different timingsmay also mean operating one image analysis engine in a period when theimage analysis is performed on the image data shot in the environment ofthe sunny weather and operating another image analysis engine in aperiod when the image analysis is performed on the image data shot inthe environment of the cloudy or rainy weather. Operating the pluralityof image analysis engines at different timings may also mean operating adifferent image analysis engine for each reference interval (as oneexample, one minute). Operating the plurality of image analysis enginesat different timings may also mean operating a different image analysisengine such that the target objects are switched during a period whenthe same (or similar) objects are included in the image data. Switchingthe target object during a period when the same (or similar) objects areincluded in the image data may mean, as one example, switching thetarget object between a face and an action during a period when a personis included in the image data as the object, or may mean switching thetarget object between a car model and a license plate during a periodwhen a car is included in the image data as the object.

At least one of the target object or the shooting condition of the imagedata appropriate for the image analysis may be different among theplurality of image analysis engines. The shooting condition of the imagedata may be the condition determined by the monitoring camera 3 whichperforms the shooting or may be the condition determined by theinstallation situation of the monitoring camera 3 or the timing ofshooting. For example, the shooting condition of the image data may beat least one of the performance of the monitoring camera 3 whichperforms the shooting (such as the number of pixels, a frame rate, orthe like), the application (indoor, outdoor, or the like), the shootingdirection (the bearing, the orientation relative to the light source,the orientation relative to the target object which is possibly includedin the image data, or the like), the installed location, (the address,the GPS data, the position relative to the target object which ispossibly included in the image data, indoor, outdoor, or the like), thenumber of target objects possibly included in the image data (single ormultiple), the situation of the target object possibly included in theimage data (moving, stopped, or the like), the hours of the shooting,the situation of the shot region at a time of the shooting (the colorand brightness of the illumination, the weather, or the like), or thelike.

Note that the image analysis engine may be a model obtained through themachine learning such as the deep learning as one example, but it is notlimited thereto.

[1. 2. 3. Communication Unit 45]

The communication unit 45 communicates the file server 5 or the like viathe communication network 10.

The communication unit 45 is one example of the transmitting unit andtransmits to the file server 5 the metadata of the image data in whichthe feature data is generated. If the feature data is generated for theplurality of image data obtained by the plurality of monitoring cameras3, the communication unit 45 may transmit the metadata for each of theplurality of monitoring cameras 3. The communication unit 45 maytransmit the metadata every time the content of the feature datachanges.

Herein, the metadata may be the data indicating the content and locationof the corresponding image data and may include feature data and atleast one of the shooting time of the image data in which the featuredata is generated or the address of the image data within the imagestorage unit 40.

If the metadata includes the shooting time, the metadata may furtherinclude a camera ID and a device ID. Furthermore, if the metadataincludes the address of the image data within the image storage unit 40,the address may further indicate the address of the device 4 within thesystem 1.

The communication unit 45 may receive the image analysis enginetransmitted from the delivery server 6 and supply it to the analyzingunit 41.

Furthermore, the communication unit 45 may transmit the thumbnail imageof the image data obtained by each of the monitoring cameras 3 to thefile server 5 or the monitoring terminal 7 at periodic timings.Furthermore, the communication unit 45 may transmit the image datastored in the image storage unit 40 to the file server 5 before theimage data is overwritten. In this case, the communication unit 45 maycompress and transmit the image data.

[1. 3. File Server 5]

The file server 5 is accessible to another apparatus (in the presentembodiment, the monitoring terminal 7 or the mobile terminal 8, as oneexample) different from the device 4 and shares the stored file withother apparatuses. The file server 5 may be simultaneously accessible toa plurality of apparatuses without increasing the communication cost.The file server 5 has a storage unit 52.

The storage unit 52 stores the metadata transmitted from the device 4.The storage unit 52 may further store the image data corresponding tothe metadata in association with the metadata.

The storage unit 52 may further store the image data of the thumbnailimage transmitted from the device 4 or the image data transmitted by thedevice 4 before deleted due to overwriting in association with thedevice ID of the transmission source. The storage unit 52 may have alarger capacity than the image storage unit 40.

[1. 4. Delivery Server 6]

The delivery server 6 delivers the image analysis engine to each of thedevices 4. The delivery server 6 has an analysis engine storage unit 61,a shooting condition storage unit 62, a detection target storage unit63, a setting unit 64, and a supplying unit 65.

[1. 4. 1. Analysis Engine Storage Unit 61]

The analysis engine storage unit 61 stores a plurality of image analysisengines. At least one of the target object or the shooting condition ofthe image data appropriate for the image analysis may be different amongthe plurality of image analysis engines. If the image analysis engine isgenerated by machine learning, each image analysis engine stored in theanalysis engine storage unit 61 may be updated through the learning ofwhether the analysis result is correct.

[1. 4. 2. Shooting Condition Storage Unit 62]

For each of the plurality of devices 4, the shooting condition storageunit 62 stores the shooting condition of the image data obtained by thedevice 4. If the shooting condition changes, the shooting conditionstorage unit 62 may store a plurality of shooting conditions. The storedshooting condition may be set by the operator.

Alternatively, the shooting condition to be stored may be set based onthe result of the image analysis performed on the image data output fromthe device 4 using each image analysis engine within the analysis enginestorage unit 61. For example, the image analysis may be performed on theimage data from one device 4 using two or more image analysis enginesfor each of which a shooting condition of the image data appropriate forthe image analysis and the same target object, among the plurality ofimage analysis engines within the analysis engine storage unit 61 isdifferent. Then, the shooting condition corresponding to the imageanalysis engine which has detected the largest number of the targetobjects as a result of the image analysis may be set as the shootingcondition of the image data obtained by the one device 4. The image datawhich is the target for the image analysis may be the image data such asthe thumbnail image stored in, for example, the file server 5.

[1. 4. 3. Detection Target Storage Unit 63]

For each of the plurality of devices 4, the detection target storageunit 63 stores the target object which is set for the device 4. Thedetection target storage unit 63 may store two or more target objectsfor one device 4. The target object to be stored may be set by theoperator or may be optionally changed.

[1. 4. 4. Setting Unit 64]

The setting unit 64 sets the operating condition of each image analysisengine in each of the devices 4. The operating condition of the imageanalysis engine may be the condition of determining the timing at whichthe image analysis engine performs the image analysis or may be theshooting condition of the image data which is the target of the imageanalysis. As one example, the operating condition may be that the imagedata is shot in the daytime or the nighttime. In this case, the imageanalysis engine performs the image analysis on the image data shot inthe daytime or the nighttime.

For the analyzing unit 41 of each of the devices 4, the setting unit 64may set the operating condition of the image analysis engine used by theanalyzing unit 41 via the supplying unit 65. Thereby, the analyzing unit41 of each of the devices 4 causes each of the image analysis engines tooperate with the set operating condition.

[1. 4. 5. Supplying Unit 65]

The supplying unit 65 supplies, to at least one device 4, any of theplurality of image analysis engines stored in the analysis enginestorage unit 61. The supplying unit 65 may supply to the device 4 theimage analysis engine corresponding to at least one of the shootingcondition of the image data obtained by the device 4 or the targetobjects which are set in the device 4.

The supplying unit 65 may refer to the shooting condition storage unit62 to identify the shooting condition of the image data obtained by thedevice 4. The supplying unit 65 may refer to the detection targetstorage unit 63 to identify the target object which is set by the device4.

The supplying unit 65 may supply the same image analysis engines to eachof two or more of the devices 4 which obtain the image data having atleast partially the same shooting condition, among the plurality ofdevices 4 included in the system 1. Thereby, for example, if each of thetwo or more of the devices 4 obtains the image data shot in the samehours, the image analysis engine appropriate for the image data shot inthe hours is supplied to each of the two or more of the devices 4.

The supplying unit 65 may supply the same image analysis engine to eachof the two or more of the devices 4 having the same target object, amongthe plurality of devices 4 included in the system 1. The supplied imageanalysis engine may be received by the communication unit 45 of thedevice 4 and used by the analyzing unit 41.

[1. 5. Monitoring Terminal 7]

The monitoring terminal 7 performs the monitoring of the monitoredregion. The monitoring terminal 7 has a selecting unit 75, acommunication unit 76, and a display control unit 71.

[1. 5. 1. Selecting Unit 75]

The selecting unit 75 selects any one of the metadata stored in the fileserver 5 in response to the operation by the operator. The selectingunit 75 may have an input unit 751 and a searching unit 752, and mayselect the metadata detected by the input unit 751 and the searchingunit 752 and supply it to the communication unit 76.

The input unit 751 inputs the feature data of the object in response tothe operation by the operator. The input unit 751 may supply the inputfeature data to the searching unit 752.

Herein, the feature data input from the operator may be the dataindicating the feature of the object which is possibly shot by any ofthe monitoring cameras 3. For example, if police or the like requiresinformation on a particular person or vehicle to be provided, thefeature data input by the operator may be the data indicating thefeature of the person or vehicle.

The searching unit 752 searches for the metadata including the inputfeature data, among a plurality of metadata stored in the file server 5(in the present embodiment, the data including the feature data and atleast one of the shooting time or the address of the image data, as oneexample).

[1. 5. 2. Communication Unit 76]

The communication unit 76 is one example of the reading unit and readsfrom the device 4 the image data corresponding to the selected metadata.The communication unit 76 may supply the read image data to the displaycontrol unit 71.

Herein, if the metadata includes the shooting time, the image datacorresponding to the metadata may be the image data shot at the shootingtime or may be the image data shot in the reference duration includingthe shooting time. In the present embodiment, the reference duration istwo minutes as one example, but it may also be a different duration suchas 10 minutes or one day.

If the metadata further includes a camera ID and a device ID, the imagedata corresponding to the metadata may be the image data shot by themonitoring camera 3 corresponding to the camera ID among the image datastored in the device 4 corresponding to the device ID. If the metadataincludes the device ID and does not include the camera ID, the imagedata corresponding to the metadata may be the image data shot by each ofthe monitoring cameras 3 among the image data stored in the device 4corresponding to the device ID. If the metadata does not include thedevice ID or the camera ID, the image data corresponding to the metadatamay be the image data shot by each of the monitoring cameras 3 among theimage data stored in each of the devices 4.

Furthermore, if the metadata includes the address of the image datawithin the image storage unit 40, the image data corresponding to themetadata may be the image data within the image storage unit 40 of thedevice 4 indicated by the address or may be the image data shot withinthe reference duration including the image data.

Furthermore, the communication unit 76 is one example of the storagecontrol unit and causes the read image data to be stored in the fileserver 5. The communication unit 76 may cause the read image data to bestored in the file server 5 in association with the metadata used forthe reading.

[1. 5. 3. Display Control Unit 71]

The display control unit 71 displays the image data read by thecommunication unit 76.

If no metadata is selected by the selecting unit 75 and no image data isread by the communication unit 76 and supplied to the display controlunit 71, the display control unit 71 may list the thumbnail image of theimage data obtained from each of the monitoring cameras 3 via each ofthe devices 4.

Furthermore, in response to a thumbnail image of any of the monitoringcameras 3 being selected by the operator while the thumbnail images arelisted, the display control unit 71 may display the image data obtainedby the monitoring camera 3 by reading it from the image storage unit 40of the device 4.

[1. 6, Mobile Terminal 8]

Each of the mobile terminals 8 is carried by a security guard of asecurity company in each area. The mobile terminal 8 may be able toaccess the file server 5.

[1. 7. Advantage Obtained Through the System 1]

According to the delivery server 6 of the above-described system 1, thedevice 4 is supplied with the image analysis engine corresponding to atleast one of the shooting condition of the image data obtained by thedevice 4 or the target object which is set for the device 4, among theplurality of image analysis engines stored in the analysis enginestorage unit 61. Thereby, the device 4 can perform the image analysis byusing an appropriate analysis engine. For example, if the shootingconditions of the image data are different among the monitoring cameras3, the device 4 can perform the image analysis by using the imageanalysis engine corresponding to the shooting condition of themonitoring camera 3. Furthermore, if the target object is changed, thedevice 4 can perform the image analysis by using the image analysisengine corresponding to the target object after the change.

Furthermore, since the same image analysis engines are supplied to eachof two or more of the devices 4 which obtain the image data having atleast partially the same shooting condition, the image analysis can beperformed with the same image analysis engine on the image data which isobtained by each of the devices 4 and has at least partially the sameshooting condition.

Furthermore, since the same image analysis engines are supplied to eachof two or more of the devices 4 having the same target objects, each ofthe devices 4 which should detect the same target object can perform theimage analysis by using the same image analysis engine.

Furthermore, since the operating condition of each image analysis enginewithin the device 4 is set by the setting unit 64, the device 4 forwhich any operating condition is set in advance can perform the imageanalysis with any operating condition.

Furthermore, the device 4 of the above-described system 1 performs theimage analysis on the image data obtained by the monitoring camera 3,generates the feature data indicating the feature of a preset object inresponse to the detection of the object, and transmits to the fileserver 5 the metadata including the feature data and at least one of theshooting time of the image data in which the feature data is generatedor the address of the image data within the image storage unit 40. Thus,the metadata which is stored in the file server 5 can be used as anindex for searching for the image data. Thereby, the access to the imagedata in which the feature data is generated among the image data withinthe device 4 is facilitated and an object which was present in themonitored region in the past can be easily found.

Furthermore, since the metadata is transmitted for each of the pluralityof monitoring cameras 3, the object which was present in the monitoredregion can be more easily found.

Furthermore, since the metadata is transmitted every time the content ofthe feature data changes, the metadata is transmitted every time theobject is switched. Thus, the access to the image data for each of theobjects can be facilitated and the object which was present in themonitored region can be more easily found.

Furthermore, since the feature data indicating at least one of thefeatures of the face within the image, the number of the vehicle withinthe image, or the car model of the vehicle within the image is generatedand is included in the metadata, a person or a car which was present inthe monitored region can be more easily found. Similarly, in a casewhere the feature data indicating, as the feature data of the targetobject, the feature of the whole body or clothes of a person, a vehicle(a vehicle in motion or a vehicle which is stopped), a car color, aparticular object (a bag, a weapon), or the like is generated andincluded in the metadata, the target object which was present in themonitored region can also be more easily found.

Furthermore, since a plurality of image analysis engines operates inparallel or at different timings in the analyzing unit 41, a targetobject which is present in the monitored region can be assured to bedetected.

Furthermore, according to the monitoring terminal 7 of theabove-described system 1, any one of the metadata stored in the fileserver 5 is selected in response to the operation by the operator, andthe image data corresponding to the selected metadata is read anddisplayed. Therefore, any image data in which the feature data isgenerated can be easily checked.

Furthermore, when the feature data of the object is input in response tothe operation by the operator, the metadata including the input featuredata among the plurality of metadata stored in the file server 5 isdetected and displayed and thus the image data of the objectcorresponding to any feature data can be easily checked.

Furthermore, since the read image data is stored in the file server 5,the image data corresponding to the selected metadata can be collectedin the file server 5. In addition, the image data in which the featuredata is generated is prevented from being deleted due to overwriting orthe like in the device 4, and the image data can be assured to be saved.

[2. Shooting Condition Storage Unit 62]

FIG. 2 shows the stored content of the shooting condition storage unit62. In the present embodiment, as one example, the shooting conditionstorage unit 62 stores, for each of the plurality of devices 4, thedevice ID (for example, such as “aaa1”) of the device 4, the camera ID(for example, such as “c-01”) of the monitoring camera 3 connected tothe device 4, and the shooting condition of the image data (for example,the monitoring camera 3 is for the indoor application and the installedlocation is indoor, or the like) in association with each other.

[3. Detection Target Storage Unit 63]

FIG. 3 shows the stored content of the detection target storage unit 63.In the present embodiment, as one example, the detection target storageunit 63 stores, for each of the plurality of devices 4, the device ID(for example, such as “aaa1”) of the device 4 and the target object (forexample, such as “a human face”) which is set for the device 4 inassociation with each other.

[4. Operation Of The System 1]

[4. 1. Delivery of the Image Analysis Engine]

FIG. 4 shows the operation related to the delivery of the image analysisengine. The system 1 delivers the image analysis engine to each of thedevices 4 by performing the processes of steps S11 to S13. Note that atthe start of this operation the analysis engine storage unit 61 of thedelivery server 6 may have already stored the plurality of imageanalysis engines for each of which at least one of a shooting conditionof the image data appropriate for the image analysis or a target objectto be detected is different. Each image analysis engine may include theinformation indicating the detection object to be detected by the imageanalysis and the shooting condition of the image data appropriate forthe image analysis.

At step S11, the supplying unit 65 supplies to the device 4 any of theplurality of image analysis engines stored in the analysis enginestorage unit 61. The supplying unit 65 may refer to at least one of theshooting condition storage unit 62 or the detection target storage unit63 to supply to the device 4 the image analysis engine corresponding toat least one of the shooting condition of the image data obtained by thedevice 4 or the target objects which is set in the device 4. Note thatin this step S11 the supplying unit 65 may receive from the setting unit64 the operating condition of the image analysis engine to be suppliedand further supply it to the device 4.

The supplying unit 65 may update the image analysis engine in the device4 with the image analysis engine to be supplied or may add into thedevice 4 the image analysis engine to be supplied. For example, thesupplying unit 65 may instruct the analyzing unit 41 of the device 4 toupdate or add the image analysis engine. If the supplying unit 65instructs the update of the image analysis engine, it may designate theimage analysis engine to be updated among the one or more image analysisengines used by the analyzing unit 41.

The supplying unit 65 may supply the image analysis engine in responseto the operation by the operator. For example, if at least one of theupdate or the addition of the image analysis engine stored in theanalysis engine storage unit 61 is performed in response to theoperation by the operator, the supplying unit 65 may supply the imageanalysis engine to the device 4. In this case, the supplying unit 65 maydetect the detection object of each image analysis engine which isupdated or added and the shooting condition of the image dataappropriate for the image analysis and supply the image analysis engineto the device 4 corresponding to at least one of these detection objector shooting condition.

Furthermore, if the information of at least one of the detection targetstorage unit 63 or the shooting condition storage unit 62 in response tothe operation by the operator is changed, the supplying unit 65 maysupply the image analysis engine. In this case, the supplying unit 65may supply the image analysis engine corresponding to the detectionobject after the change, to each of the devices 4 corresponding to thedetection object changed in the detection target storage unit 63.Furthermore, the supplying unit 65 may supply the image analysis enginecorresponding to the shooting condition after the change, to each of thedevices 4 corresponding to the shooting condition changed in theshooting condition storage unit 62.

At step S13, the analyzing unit 41 of the device 4 obtains the imageanalysis engine via the communication unit 45. If instructed to updatethe image analysis engine by the supplying unit 65, the analyzing unit41 updates any of the image analysis engines with the supplied imageanalysis engine. If instructed to add the image analysis engine by thesupplying unit 65, the analyzing unit 41 adds the supplied imageanalysis engine to the target for use. If receiving the operatingcondition of the image analysis engine from the supplying unit 65, theanalyzing unit 41 performs setting such that the supplied image analysisengine is operated with the operating condition.

According to the above-described operation, since the image analysisengine in the device 4 is updated with the supplied image analysisengine, the device 4 can use a new image analysis engine to perform theimage analysis even in the presence of the limitation on the storagearea in the device 4.

Furthermore, since the supplied image analysis engine is added into thedevice 4, the device 4 can use a plurality of image analysis engines toperform the image analysis.

[4. 2. Monitoring]

FIG. 5 shows an operation related to the monitoring. The system 1performs the monitoring of the monitored region by performing theprocesses of steps S51 to S69. Note that the device 4 may cause theimage data obtained by the monitoring camera 3 to be stored in the imagestorage unit 40 during the operation.

At step S51, the analyzing unit 41 of the device 4 uses one or moreimage analysis engines to perform the image analysis on the obtainedimage data and, in response to the detection of a target object,generates the feature data indicating the feature of the object.

At step S53, the communication unit 45 generates the metadata includingthe feature data and at least one of the shooting time of the image datain which the feature data is generated or the address of the image datawithin the image storage unit 40 and transmits it to the file server 5.

At step S55, the storage unit 52 of the file server 5 stores thetransmitted metadata.

At step S57, the selecting unit 75 of the monitoring terminal 7 selectsany one of the metadata stored in the file server 5 in response to theoperation by the operator. In the present embodiment, as one example,the selecting unit 75 may select the metadata including the feature datainput by the operator via the input unit 751.

At step S61, the communication unit 76 reads from the image storage unit40 of the device 4 the image data corresponding to the selectedmetadata. In response to this, at step S63, the device 4 may provide themonitoring terminal 7 with the access to the image storage unit 40.

At step S65, the display control unit 71 of the monitoring terminal 7displays the read image data. Note that the processes at steps S57, S61,and S65 may also be performed by the mobile terminal 8. In this case,the device 4 provides the mobile terminal 8 with the access to the imagestorage unit 40, and the mobile terminal 8 displays the read image data.Thereby, the situation of the monitored region can be checked by asecurity guard.

At step S67, the communication unit 76 transmits the read image data tothe file server 5 in association with the metadata. The communicationunit 76 may perform the transmission in response to the instruction bythe operator who has checked the displayed image.

Then, at step S69, the storage unit 52 of the file server detects thesame metadata as the received metadata, among the plurality of metadatawhich has been stored, and stores the received image data in associationwith the metadata.

[5, The Modified Example of an Embodiment]

Note that in the above-described embodiment it has been described thatthe monitoring terminal 7 and the delivery server 6 are distinctdevices, but they may also be the same device.

Furthermore, it has been described that the delivery server 6 has thesetting unit 64 which sets the operating condition of each imageanalysis engine in each of the devices 4, but it may have an instructingunit (not shown) which instructs each image analysis engine in thedevice 4 to perform the image analysis. In this case, the image analysisengine can perform the image analysis each time depending on thesituation.

Furthermore, it has been described that the delivery server 6 has theshooting condition storage unit 62, the detection target storage unit63, and the setting unit 64 (or the instructing unit), but it may nothave at least one of these units. For example, if the delivery server 6does not have the setting unit 64 (or the instructing unit), theoperating condition of the image analysis engine may be set at the sideof the device 4.

Furthermore, it has been described that the image data corresponding tothe metadata selected by the monitoring terminal 7 is read from thedevice 4 and is saved in the file server 5, but the deletion of theimage data corresponding to the metadata may be prevented throughanother method. For example, in order to prevent the image data in whichthe feature data is generated from being overwritten and deleted, theanalyzing unit 41 of the device 4 may lock the image data within theimage storage unit 40.

Furthermore, it has been described that the device 4 stores the imagedata obtained by the monitoring camera 3 in the image storage unit 40,but the image data may be stored in response to the detection of thetarget object by the analyzing unit 41. In this case, since the amountof the image data saved in the device 4 can be reduced, the device 4 maynot transmit the metadata to the file server 5.

Furthermore, it has been described that the device 4 has the singleanalyzing unit 41, but it may also have a plurality of analyzing units41 of the same type. In this case, the image analysis can be performedin parallel at a fast speed.

Also, various embodiments of the present invention may be described withreference to flowcharts and block diagrams. Blocks may represent (1)steps of processes in which operations are performed or (2) sections ofapparatuses responsible for performing operations. Certain steps andsections may be implemented by dedicated circuitry, programmablecircuitry supplied with computer-readable instructions stored oncomputer-readable media, and/or processors supplied withcomputer-readable instructions stored on computer-readable media.Dedicated circuitry may include digital and/or analog hardware circuitsand may include integrated circuits (IC) and/or discrete circuits.Programmable circuitry may include reconfigurable hardware circuitsincluding logical AND, OR, XOR, NAND, NOR, and other logical operations,flip-flops, registers, memory elements, etc., such as field-programmablegate arrays (FPGA), programmable logic arrays (PLA), etc.

Computer-readable media may include any tangible device that can storeinstructions for execution by a suitable device, such that thecomputer-readable medium having instructions stored therein comprises anarticle of manufacture including instructions which can be executed tocreate means for performing operations specified in the flowcharts orblock diagrams. Examples of computer-readable media may include anelectronic storage medium, a magnetic storage medium, an optical storagemedium, an electromagnetic storage medium, a semiconductor storagemedium, etc. More specific examples of computer-readable media mayinclude a floppy (registered trademark) disk, a diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an electricallyerasable programmable read-only memory (EEPROM), a static random accessmemory (SRAM), a compact disc read-only memory (CD-ROM), a digitalversatile disk (DVD), a BLU-RAY (registered trademark) disc, a memorystick, an integrated circuit card, etc.

Computer-readable instructions may include assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, JAVA (registeredtrademark), C++, etc., and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages.

Computer-readable instructions may be provided to a processor of ageneral purpose computer, special purpose computer, or otherprogrammable data processing device, or to programmable circuitry,locally or via a local area network (LAN), wide area network (WAN) suchas the Internet, etc., to execute the computer-readable instructions tocreate means for performing operations specified in the flowcharts orblock diagrams. Examples of processors include computer processors,processing units, microprocessors, digital signal processors,controllers, microcontrollers, etc.

FIG. 6 shows an example of a computer 2200 in which a plurality ofaspects of the present invention may be implemented entirely orpartially. A program that is installed in the computer 2200 can causethe computer 2200 to function as or perform operations associated withapparatuses of the embodiments of the present invention or one or moresections thereof, or perform the operations or the one or more sections,and/or cause the computer 2200 to perform processes of the embodimentsof the present invention or steps of the processes. Such a program maybe executed by the CPU 2212 to cause the computer 2200 to performcertain operations associated with some or all of the blocks offlowcharts and block diagrams described herein.

The computer 2200 according to the present embodiment includes a CPU2212, a RAM 2214, a graphic controller 2216, and a display device 2218,which are mutually connected by a host controller 2210. The computer2200 also includes input/output units such as a communication interface2222, a hard disk drive 2224, a DVD-ROM drive 2226 and an IC card drive,which are connected to the host controller 2210 via an input/outputcontroller 2220. The computer also includes legacy input/output unitssuch as a ROM 2230 and a keyboard 2242, which are connected to theinput/output controller 2220 through an input/output chip 2240.

The CPU 2212 operates according to programs stored in the ROM 2230 andthe RAM 2214, thereby controlling each unit. The graphic controller 2216obtains image data generated by the CPU 2212 on a frame buffer or thelike provided in the RAM 2214 or in itself, and causes the image data tobe displayed on the display device 2218.

The communication interface 2222 communicates with other electronicdevices via a network. The hard disk drive 2224 stores programs and dataused by the CPU 2212 within the computer 2200. The DVD-ROM drive 2226reads the programs or the data from the DVD-ROM 2201, and provides thehard disk drive 2224 with the programs or the data via the RAM 2214. TheIC card drive reads programs and data from an IC card, and/or writesprograms and data into the IC card.

The ROM 2230 stores therein a boot program or the like executed by thecomputer 2200 at the time of activation, and/or a program depending onthe hardware of the computer 2200. The input/output chip 2240 may alsoconnect various input/output units via a parallel port, a serial port, akeyboard port, a mouse port, or the like to the input/output controller2220.

A program is provided by computer readable media such as the DVD-ROM2201 or the IC card. The program is read from the computer readablemedia, installed into the hard disk drive 2224, RAM 2214, or ROM 2230,which are also examples of computer readable media, and executed by theCPU 2212. The information processing described in these programs is readinto the computer 2200, resulting in cooperation between a program andthe above-mentioned various types of hardware resources. An apparatus ormethod may be constituted by realizing the operation or processing ofinformation in accordance with the usage of the computer 2200.

For example, when communication is performed between the computer 2200and an external device, the CPU 2212 may execute a communication programloaded onto the RAM 2214 to instruct communication processing to thecommunication interface 2222, based on the processing described in thecommunication program. The communication interface 2222, under controlof the CPU 2212, reads transmission data stored on a transmissionbuffering region provided in a storage medium such as the RAM 2214, thehard disk drive 2224, the DVD-ROM 2201, or the IC card, and transmitsthe read transmission data to a network or writes reception datareceived from a network to a reception buffering region or the likeprovided on the storage medium.

In addition, the CPU 2212 may cause all or a necessary portion of a fileor a database to be read into the RAM 2214, the file or the databasehaving been stored in an external storage medium such as the hard diskdrive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), the IC card, etc.,and perform various types of processing on the data on the RAM 2214. TheCPU 2212 may then write back the processed data to the external storagemedium.

Various types of information, such as various types of programs, data,tables, and databases, may be stored in the storage medium to undergoinformation processing. The CPU 2212 may perform various types ofprocessing on the data read from the RAM 2214, which includes varioustypes of operations, processing of information, condition judging,conditional branch, unconditional branch, search/replace of information,etc., as described throughout this disclosure and designated by aninstruction sequence of programs, and writes the result back to the RAM2214. In addition, the CPU 2212 may search for information in a file, adatabase, etc., in the storage medium. For example, when a plurality ofentries, each having an attribute value of a first attribute associatedwith an attribute value of a second attribute, are stored in the storagemedium, the CPU 2212 may search for an entry matching the conditionwhose attribute value of the first attribute is designated, from amongthe plurality of entries, and read the attribute value of the secondattribute stored in the entry, thereby obtaining the attribute value ofthe second attribute associated with the first attribute satisfying thepredetermined condition.

The above-explained program or software modules may be stored in thecomputer readable media on or near the computer 2200. In addition, astorage medium such as a hard disk or a RAM provided in a server systemconnected to a dedicated communication network or the Internet can beused as the computer readable media, thereby providing the program tothe computer 2200 via the network.

While the embodiments of the present invention have been described, thetechnical scope of the invention is not limited to the above-describedembodiments. It is apparent to persons skilled in the art that variousalterations and improvements can be added to the above-describedembodiments. It is also apparent from the scope of the claims that theembodiments added with such alterations or improvements can be includedin the technical scope of the invention.

The operations, procedures, steps, and stages of each process performedby an apparatus, system, program, and method shown in the claims,embodiments, or diagrams can be performed in any order as long as theorder is not indicated by “prior to,” “before,” or the like and as longas the output from a previous process is not used in a later process.Even if the process flow is described using phrases such as “first” or“next” in the claims, embodiments, or diagrams, it does not necessarilymean that the process must be performed in this order.

EXPLANATION OF REFERENCES

1 system,

3 monitoring camera,

4 device,

5 file server,

6 delivery server,

7 monitoring terminal,

8 mobile terminal,

10 communication network,

40 image storage unit,

41 analyzing unit,

45 communication unit,

52 storage unit,

61 analysis engine storage unit,

62 shooting condition storage unit,

63 detection target storage unit,

64 setting unit,

65 supplying unit,

71 display control unit,

75 selecting unit,

76 communication unit,

751 input unit,

752 searching unit,

2200 computer,

2201 DVD-ROM,

2210 host controller,

2212 CPU,

2214 RAM,

2216 graphics controller,

2218 display device,

2220 input/output controller,

2222 communication interface,

2224 hard disk drive,

2226 DVD-ROM drive,

2230 ROM,

2240 input/output chip,

2242 keyboard

What is claimed is:
 1. A device comprising: an image storage unit whichis configured to store image data obtained by a monitoring camera; ananalyzing unit which is configured to perform image analysis on theimage data and, in response to a detection of a target object to bedetected among objects which are possibly included in the image data,generate feature data indicating a feature of the target object; and atransmitting unit which is configured to transmit metadata including thefeature data and at least one of a shooting time of an image data inwhich the feature data is generated or an address of the image datawithin the image storage unit to a file server which is accessible toanother apparatus different from the device.
 2. The device according toclaim 1, wherein the transmitting unit is configured to transmit themetadata every time a content of the feature data changes.
 3. The deviceaccording to claim 1, wherein the image storage unit is configured tostore each image data obtained by a plurality of monitoring cameras, andthe transmitting unit is configured to transmit the metadata for each ofthe plurality of monitoring cameras.
 4. The device according to claim 1,wherein the feature data indicates at least one of a feature of a facewithin an image, a number of vehicle within an image, or a car model ofa vehicle within an image.
 5. The device according to claim 1, whereinthe analyzing unit is configured to operate a plurality of imageanalysis engines in parallel or at different timings.
 6. A systemcomprising: the device according to claim 1; the file server which isconfigured store the metadata transmitted from the device; and amonitoring terminal which is configured to be connected to the deviceand the file server, wherein the monitoring terminal includes: aselecting unit which is configured to select any one of the metadatastored in the file server in response to an operation by an operator; areading unit which is configured to read, from the device, image datacorresponding to the metadata which is selected; and a display controlunit which is configured to display the image data read by the readingunit.
 7. The system according to claim 6, wherein the selecting unitincludes: an input unit which is configured to input feature data of anobject corresponding to an operation of an operator; and a searchingunit which is configured to search for metadata including the featuredata which is input, among the plurality of metadata stored in the fileserver, wherein the selecting unit is configured to select the metadatadetected by the searching unit.
 8. The system according to claim 6,wherein the monitoring terminal further includes a storage control unitwhich is configured to cause the image data read by the reading unit tobe stored in the file server.
 9. The system according to claim 6,further comprising a delivery server which is configured to be connectedto the device, wherein the delivery server includes: an analysis enginestorage unit which is configured to store a plurality of image analysisengines for each of which at least one of a shooting condition of theimage data appropriate for image analysis or a feature of a targetobject to be detected among objects which are possibly included in theimage data is different; and a supplying unit which is configured tosupply to the device an image analysis engine corresponding to at leastone of the shooting condition of the image data obtained by the deviceor the target object which is set for the device, among the plurality ofimage analysis engines.
 10. A method performed in a device, comprising:storing image data obtained by a monitoring camera in a storage device;performing image analysis on the image data and, in response to adetection of a target object to be detected among objects which arepossibly included in the image data, generating feature data indicatinga feature of the target object; and transmitting metadata including thefeature data and at least one of a shooting time of an image data inwhich the feature data is generated or an address of the image datawithin the storage device to a file server accessible to anotherapparatus different from the device.
 11. A storage medium which isconfigured to record a program which causes a computer of a device toserve as: an image storage unit which is configured to store image dataobtained by a monitoring camera; an analyzing unit which is configuredto perform image analysis on the image data and, in response to adetection of a target object to be detected among objects which arepossibly included in the image data, generate feature data indicating afeature of the target object; and a transmitting unit which isconfigured to transmit metadata including the feature data and at leastone of a shooting time of the image data in which the feature data isgenerated or an address of the image data within the image storage unitto a file server which is accessible to another apparatus different fromthe device.